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Preface 


he current national debate about higher 
education has focused heavily on varying 
interpretations of the U.S. higher education 
system’s accountability structures. Much of that 
debate has emphasized undergraduate learning 
and the specific issues of student engagement and 
outcomes. But relatively little attention has been 
paid at either the state or federal level to questions 
about community college accountability systems, 
and specifically whether statewide accountability 
systems are helping or hurting efforts to improve 
performance by community colleges. 

This working paper evaluates the accountability 
systems in several key states in order to determine 
whether — and how — these state systems consider 


the unique situation of community colleges. The 
goal of the paper is to better understand the 
interconnections between the wholesale issues 
about accountability systems and the relatively 
more retail questions about whether these issues 
actually resonate in a meaningful way in the sector 
that educates the largest and most diverse number 
of students in American higher education. We hope 
this working paper will be a useful addition to the 
debates about accountability occurring in states 
across the nation, as well as inform the national 
discussions about how such systems might be 
linked to one another to meet national goals. 

Jamie P. Merisotis 

President, Institute for Higher Education Policy 
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Introduction 


I n the last few years, a remarkable consensus 
has begun to emerge about a new public agenda 
for postsecondary education in America. The 
imperative is to increase production, quality, and 
affordability all across the educational pipeline 
without a substantial new infusion of public 
revenues. Each sector and institution — from public 
colleges to private research universities — has a role 
to play in this new public agenda, but the single 
greatest influence on the success or failure of the 
agenda will be the public community colleges. 

Community colleges are the largest sector of higher 
education, with the greatest preponderance of 
low-income and minority student enrollments and 
the lowest average cost to students (American 
Association of Community Colleges 2004). They 
will absorb the lion’s share of projected enrollment 
growth in many states, including the megastates of 
Texas, California, and Florida. Community colleges 
can be the most cost-effective route to educational 
success for students and for the state, yet at 
the same time, because of their historically low 
completion rates, community colleges are a low- 
cost, but high-risk, option for students interested in 
obtaining a degree or certificate. 

All this means that the role of community 
colleges — and their performance within the larger 


context of state goals — needs to be front and center 
in statewide accountability measures. Accountability 
systems have become one of the most important 
vehicles for state policy in postsecondary education, 
as they report measures of institutional performance 
within the context of larger statewide goals. 

However, after taking a close look at the treatment 
of community colleges in selected statewide 
accountability systems, we think these systems still 
fail to support the imperative to increase production, 
quality, and affordability. 

Our study suggests that, in their present form, 
statewide accountability systems are not likely 
to provide state policymakers with the kind of 
information they need to identify viable options and 
make effective choices to meet state performance 
goals. Accountability systems in their present 
form can also be a distraction rather than a help 
at the institutional level. As a result, critical 
policy decisions — like whether to accommodate 
enrollment growth in community colleges rather 
than in more expensive four-year institutions — are 
often being made based on weak or nonexistent 
data. If community colleges are to take on their 
share of new enrollments and operate both 
efficiently and effectively within the context of their 
varied missions, statewide accountability structures 
will need to be reassessed. 
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Project Background 


T his working paper focuses on the relationship 
between statewide higher education 
accountability systems and the public 
community college sector — and whether such 
accountability systems are a help or a hindrance 
to improved performance by community colleges 
within the larger landscape of state postsecondary 
policy. The intent of this paper is to evaluate the 
accountability systems in several states in an effort 
to determine the extent to which these systems 
consider the unique situation of community 
colleges and yet still support state policy goals. 

By their nature, community colleges probably 
present the greatest challenge to state 
and institutional policymakers in building 
effective accountability systems. More than 
any other sector, these institutions have the 
public mandate to accomplish widely diverse 
missions — adult basic education, remedial 
education, core academic education, job training 
and certification, workforce skill development, 
continuing education, and transfer preparation. If 
accountability systems do not take into account 
this range of missions, as well as the diverse 
populations served by community colleges 
and the low levels of funding they often face, 
these institutions may find it difficult to meet 
accountability standards and may be forced to 
reduce the services they offer (Dougherty & Hong 
2005; McClenney 2005). 

For this study, we reviewed the national literature 
on postsecondary accountability and conducted 
interviews with national experts in the field. 
Throughout this review, we found considerable 
consensus about operating principles to guide 


the evaluation of accountability systems.^ This 
consensus has provided us with a conceptual 
framework to evaluate specific accountability 
systems. 

In our assessment, when statewide higher 
education accountability systems work well, they 
do so in six different areas: 

O Focus: In order to facilitate effective 
policymaking, the accountability measures 
used are organized in relation to specific goals. 
In a statewide system, measures are linked to 
broad state goals in areas such as academic 
preparation, affordable access, student 
success, quality of learning, and racial and 
economic equity. 

Q Differentiation: The accountability system 
recognizes different layers of responsibility 
for performance, and information is clustered 
appropriately for different audiences and 
decision makers. The key distinction here is 
typically between a statewide focus, with a 
legislative or gubernatorial audience, and a 
systemwide or institutional focus, aimed at 
chancellors, presidents, and deans. 

0 Contextualization: Information is made more 
meaningful by being put into context through 
benchmarking, trend analysis, or peer group 
comparisons. This contextualization recognizes 
that institutions can have very different missions 
and student populations. 

O Integrity: The data sources, and the metrics 
used, represent the best measures available, 
derive from high quality and consistently 
reported data, and are readily understandable. 


* See Zumeta 2000; Wellman 2002; Shulock 2003; Business-Higher Education Forum 2004; Leveilie 2005; National Commission on 
Accountability in Higher Education 2005. 
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0 Attention to resources: Performance information 
is combined with data about resource use 
in order to allow decision makers to make 
assessments about efficiency and effectiveness 
and to be able to judge the trade-offs between 
different investment options. 

0 Stability and usability: The accountability 
systems remain in place long enough that their 
effectiveness can be judged and policymakers 
can use the systems’ measures to make 
informed decisions. 

With these six areas in mind, we examined the 
treatment of community colleges in the statewide 
accountability systems in eight states: California, 
Florida, Kentucky, Maryland, North Carolina, 

Texas, Virginia, and Washington. These states 
were selected because they have strong community 
college systems and generally good capacities for 
data collection. Each state chosen also has an 
established or developing statewide accountability 
system incorporating community college reporting. 
As a result, these states are the most likely 
provide us with models for effective and innovative 
means of handling community colleges in their 
accountability systems. 

Moreover, the states selected for this study 
represent some of the highest growth states in 
the nation and will be looking to the community 
college sector to accommodate a significant 


share of enrollment growth. Texas, for example, 
is anticipating a substantial increase in the 
number of students graduating from high school 
each year. By 2010, this number is expected 
to increase 20 percent to more than 268,000 
annually. With Texas high school graduates 
going directly to college at a rate of more than 
53 percent, nearly 24,000 additional incoming 
students will seek places in the state’s institutions 
of higher education in 2010 (National Center for 
Higher Education Management Systems 2002). 
For states such as Texas, then, developing new 
ways of assessing community college performance 
is a pressing matter. 

Published materials were used to examine each 
state’s history of higher education accountability 
as well as their current and pending accountability 
legislation. During the summer of 2005, telephone 
interviews were conducted with representatives 
from the state higher education agency and 
from the community college system or statewide 
community college association in each of the 
eight states. These individuals were asked about 
the origins and structure of the state’s current 
accountability system with attention to goals and 
measures of performance, reporting structures, 
audience, and links to funding. Each interviewee 
was also asked to assess the effectiveness of the 
accountability system, particularly in regards to 
community colleges, and to comment on likely 
changes in the future. 
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Key Findings 


T he good news is that accountability is clearly 
on the agenda for state policymakers. Many 
states are also taking advantage of lessons 
learned from previous accountability efforts and 
are attempting to accommodate the specific needs 
and concerns of community colleges through 
separate sets of performance measures (Boswell 
2005; Ewell 2005). While community colleges 
are often at a disadvantage in their data collection 
capacities, there are ongoing efforts to improve this 
situation in a number of states. Results of these 
efforts promise a wider range of useful performance 
measures including improved measures of student 
outcomes (McClenney 2005). Nonetheless, in the 
states we surveyed for this study, we found the 
accountability systems to be quite uneven in their 
attention to the six key areas identified above. In 
particular, our study found significant problems in 
linking performance measures both to statewide 
goals for higher education and to the resources 
used in achieving those goals. 

Focus 

Higher education officials and state policymakers 
sometimes think about accountability as simply 
measuring institutional performance, but 
accountability also must examine the relationship 
between institutional performance and larger policy 
goals (Shulock 2003; Boswell 2005; National 
Commission on Accountability in Higher Education 
2005). Among the states reviewed for this study, 
there is a new emphasis on establishing statewide 
goals for higher education and on collecting 
accountability data directly related to those goals. 
While Kentucky and Maryland have, for some 
time, had statewide accountability systems that 
adhere to these practices, other states have not. At 
the time of this study, however, Texas had recently 
initiated such a system, and Florida, Virginia, and 
Washington were in the process of developing 


new accountability systems that emphasize state 
goals. Florida’s proposed accountability system 
would, in fact, cover all public education in the 
state, from pre-school to graduate school (Florida 
K-20 Education Performance Accountability Task 
Force 2004). In contrast, California and North 
Carolina still use separate accountability systems 
for community colleges and four-year institutions, 
but California has a bill before its General 
Assembly that would establish a fully statewide 
accountability system for all higher education 
(Garcia 2005). 

Nonetheless, despite the new emphasis on 
developing statewide goals for postsecondary 
education, accountability reporting still tends to 
be primarily at the system or institutional level. 
Only a few states, most notably Kentucky and 
Washington, have explicit statewide measures of 
progress towards goals. Kentucky, for example, 
aggregates the total number of degrees awarded 
by each sector to show the overall performance 
of higher education in the state while Washington 
measures the total number of degrees awarded as 
a percentage of the college-age population. Both 
states also use statewide measures in areas such 
as college affordability and K-12 performance 
in producing college-ready students (Kentucky 
Council on Postsecondary Education 2005; 
Washington Higher Education Coordinating Board 
2005). Most states, on the other hand, seem 
to assume that meeting system or institutional 
targets automatically equals statewide progress, 
which may not, in fact, be the case, especially 
if institutions play a large role in determining 
their own benchmarks (Shulock 2003). To 
achieve statewide goals, moreover, the whole 
postsecondary education system must work 
together, and without statewide measures, it is 
difficult to assess cooperation among the various 
sectors of higher education. 
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Differentiation 

A key underlying argument in favor of statewide 
accountability systems for higher education is 
that they will provide state officials with the 
data needed to make informed decisions about 
policies and budgets (National Commission on 
Accountability in Higher Education 2005). It 
seems, however, that information collected during 
the accountability process in the states reviewed 
for this study is often used by other groups, as 
well — by the state higher education agency, by 
individual institutions, and in some cases, by the 
public. This lack of differentiation means that 
many statewide accountability systems attempt 
to be all things to all people, and as a result, 
are likely to be unsatisfactory to their various 
audiences (Shulock 2003). 

Community college presidents and governing boards 
are key users of accountability reports. Community 
colleges in all states reviewed use the data for self- 
assessment and institutional planning. In Virginia, 
for example, the chancellor of the community 
college system meets with each community 
college president on an annual basis to review the 
institution’s performance in relation to the statewide 
strategic plan for higher education. The information 
from the accountability report is shared in this 
process and utilized to improve the performance of 
individual campuses (Peterson 2005). 

The information from accountability reports also is 
used by the governing or coordinating bodies for 
higher education in most states. In Kentucky, for 
instance, accountability information is used by the 
Council on Postsecondary Education on an ongoing 
basis. Whenever the data are updated, the Council 
holds meetings to discuss the implications of the 
new information. Occasionally, colleges are asked 
to provide explanations for poor performance on 
certain indicators. The accountability reports also 
inform the Council’s policymaking. For example, 
when reports showed a low transfer rate from 
Kentucky’s community colleges to its four-year 
institutions, the Council conducted research on 
the factors influencing transfer and found that 


the lack of financial support is the primary barrier 
preventing community college students from 
continuing their education at four-year institutions. 
The findings informed and influenced new policies 
on financial aid (Applegate & Noxel 2005). 

The state legislature is an important audience for 
accountability reports in all the states reviewed 
except Florida. In Florida, only the reports that 
involve performance-based budgeting are sent 
to the legislature and the governor’s office while 
accountability assessments are not (Windham 
2005). In Texas and Washington, on the other 
hand, accountability reports help to establish 
a positive relationship between community 
colleges and the legislature and also contribute 
to obtaining more state investment in community 
colleges. In Texas, accountability measures such 
as appropriated funds per full-time equivalent 
student indicate that, although the state 
appropriation has been increasing in total value, 
the increase is not enough to keep up with the 
rise in enrollment. Largely because of this kind 
of effective data reporting, the legislature has 
been very generous to the community colleges 
(S. Brown 2005). Similarly, the State Board for 
Community and Technical Colleges in Washington 
uses accountability reports as its major resource 
to lobby the state legislature. In particular, when 
the Board wanted to obtain more appropriation for 
adult education, it used data from accountability 
reports to demonstrate that inadequate funding 
has been a barrier to the delivery of higher 
education to non-traditional students (Hale, 
Yoshiwara, & Seppanen 2005). 

Another argument for statewide accountability 
systems has been the idea of transparency and 
public disclosure (Leveille 2005). Among the eight 
states reviewed for this report, all except California 
and Florida have already made accountability 
reports for community colleges available to the 
general public, and Florida plans to do so in the 
future (Windham 2005). However, despite public 
debates about the need to enable state residents to 
become informed consumers of higher education. 
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most respondents reported little expectation that 
the data will be used by anyone except higher 
education experts. 

Contextualization 

Good accountability systems provide a context for 
the data they include, allowing their audiences 
to better understand how and why certain 
institutions and sectors may perform differently 
than others. Too often, statewide accountability 
systems have tended to become ranking systems, 
comparing performance across individual 
institutions (Phillippe 2005; Shulock 2005). 

An important feature of the state accountability 
systems reviewed in this report is their attempt to 
demonstrate increased accountability for achieving 
statewide goals while still acknowledging the 
unique situation of individual institutions. States 
are working to meet this challenge by providing 
context for the data included in their accountability 
reports and by avoiding the tendency to rank 
individual institutions, which may have very 
different missions and student bodies. 

One way that states handle the diversity of 
community colleges is by treating the community 
college system as one institution for the purposes of 
statewide accountability. In Kentucky, Virginia, and 
Washington, for example, accountability goals are 
set for the community college system as a whole, a 
procedure that can help to balance any differences 
among the colleges in the system. This approach 
is a potentially valuable one because it emphasizes 
the contribution of all the community colleges in 
a state towards that state’s broad goals for higher 
education. This approach may also help dissuade 
policymakers from attempting to rank individual 
community colleges against each other, a rather 
unproductive effort given the diversity these 
colleges generally represent. 

Similarly, using individualized benchmarks to 
assess performance helps reduce the tendency to 
compare dissimilar colleges to one another (Zumeta 
2000; Dougherty & Hong 2005). In Virginia’s new 


statewide accountability system, the performance 
of the state’s institutions of higher education, 
including the Virginia Community College System, 
will be measured by objective benchmarks 
developed by the State Council of Higher Education 
for Virginia (SCHEV). When developing these 
benchmarks, which define both minimum levels of 
performance and targets for improvement, SCHEV 
will take into account the historical performance 
of an institution and negotiate the targets for 
each measure with that institution (State Council 
of Higher Education for Virginia 2005). This 
pattern can also be found in states that do not 
treat the community colleges as one institution. 

In Maryland, for example, the state’s performance 
accountability system for community colleges 
includes 29 mandatory measures along with 10 
additional optional measures. Each college sets 
its own annual benchmarks for each indicator and 
can also add to its report any additional measures 
that highlight institutional performance in its own 
context (Keller 2005). 

Peer group comparisons are yet another way to 
handle the differences among community colleges 
in a state (Zumeta 2000). In both California 
and Texas, where the accountability systems are 
designed to evaluate an individual community 
college’s effectiveness and progress, community 
colleges are grouped based on size, location, and 
mission in order to provide a basis for appropriate 
and meaningful comparisons (Perry 2005; S. 

Brown 2005). On the other hand, a unique feature 
of Kentucky’s accountability system is that it allows 
community colleges to compare their progress on 
a particular benchmark to high performers in other 
states and to explore educational practices that may 
contribute to the effectiveness of high-performing 
peer institutions (Applegate & Noxel 2005). 

Finally, nearly every state allows its educational 
institutions, whether individual community 
colleges or the community college system as a 
whole, to provide contextual information to explain 
any problems or inconsistencies, an approach that 
can help to recognize the diversity of institutions 
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(Wellman 2002). In most cases, the data for the 
statewide accountability reports are gathered by 
the state higher education agency and then sent 
to the individual institutions for review. At that 
point, the institutions can correct any problems 
with the data and add their comments. In Texas, 
for instance, where accountability results are 
published using an interactive web-based system, 
each community college is offered the opportunity 
to explain their performance, and these 
explanations are provided on the accountability 
website, unedited, along with the performance 
indicators themselves (S. Brown 2005). 

Integrity 

Without good quality and consistent data, 
accountability systems cannot provide crucial 
information to state policymakers. For community 
colleges, in particular, collecting the necessary 
data for accountability reporting can sometimes 
be a difficult hurdle to overcome. Few community 
colleges have large enough budgets to pay for up-to- 
date database technology and full-time institutional 
researchers to analyze the data. Accountability 
reporting requirements can, in fact, be a financial 
burden for some community colleges, and the data 
they produce may be inconsistent or of poor quality 
(Boswell 2005; Ewell 2005). Moreover, not all 
states have the ability to match community college 
data with data collected by four-year institutions 
(Ewell 2005). Student-unit record systems, 
considered a necessity for effective accountability 
systems by many experts, are also not available in 
every state (McClenney 2005; Phillippe 2005). 

While the problem of collecting quality data 
remains a crucial issue for many statewide 
accountability systems, the states reviewed for this 
study all report that they are able to collect good 
and consistent data from their community colleges, 
in part because we deliberately selected large 
states with strong community college systems and 
active accountability efforts. As a result, the data 
collection capacities in these states are exemplary. 
All eight states studied have student-unit record 


systems that allow them to make assessments of 
student transfer rates, at least within the state, 
and to use data matching to get information on 
job placement from other state agencies. Equally 
important is the fact that all of these states have, 
to some extent, centralized their data collection 
and analysis. While individual institutions must still 
collect data on their students, most of the analysis 
of this data is done elsewhere — at the community 
college system headquarters in California, North 
Carolina, and Washington and at the state higher 
education coordinating agency in the remaining 
states. This centralization allows for effective use of 
resources and expert analysis of the data collected. 

Just as is it is important for statewide 
accountability systems to have quality data, it 
is also necessary for them to have performance 
measures that accurately capture the contributions 
made by the various sectors of higher education. 
Measures that are appropriate for four-year 
institutions may not be appropriate for community 
colleges, and effective accountability systems 
must confront this fact. It is clear that the states 
reviewed in this study are making an effort to tailor 
their statewide accountability systems to account 
for the differences between community colleges 
and four-year institutions. Of the eight states 
analyzed, the accountability systems in seven 
have a set of indicators used solely for community 
colleges, either as part of a separate system for 
evaluating community colleges as in California and 
North Carolina or as a separate set of indicators 
within a comprehensive statewide accountability 
system. The eighth state, Virginia, uses similar 
indicators for both four-year institutions and the 
community college system but exempts community 
colleges from indicators measuring research 
funding and transfers from two-year schools (State 
Council of Higher Education for Virginia 2005). 

Nonetheless, despite these efforts, there was 
still considerable concern among the experts 
and practitioners interviewed that the specific 
performance measures used for community 
colleges in these accountability systems remain 
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insufficient. If performance measures are to 
provide adequate information to policymakers, 
these measures must be valid assessments of 
institutional and sector performance, be easily 
understandable to their audiences, and reflect the 
full range of contributions community colleges 
make to higher education in a given state. In the 
states we examined, however, the measures used 
were often overly complex and frequently did not 
reflect the full range of functions that community 
colleges perform to help achieve statewide goals 
for higher education. 

Measures of Access 

Increasing access for historically underrepresented 
groups is a crucial goal for higher education, and 
most states in this study recognize the importance 
of this goal in their accountability measures for 
community colleges. Only North Carolina has no 
measure of participation by underrepresented 
groups as one of the state’s core accountability 
measures, although this measure is included 
among the broader contextual measures the 
community college system provides in their 
accountability reports (North Carolina Community 
College System 2005). Texas, California, 
and Washington examine community college 
enrollments by gender, age, and race/ethnicity 
and, in Washington, by first-generation college 
student and Pell Grant recipient status (Texas 
Higher Education Coordinating Board 2005; 
California Community Colleges Board of Governors 
2005; Washington Higher Education Coordinating 
Board 2005). Several other states also look 
at participation rates for specific categories of 
underrepresented students, such as minority 
students, students receiving need-based aid, or 
students who are GED recipients. 

On the other hand, fewer states in this study 
used measures that clarify student needs and the 
success of community colleges in meeting them. 
Only half of the eight states studied consider the 
affordability of community colleges. Kentucky, for 
instance, assesses college cost as a percentage 
of the state’s median household income, while 


Maryland looks at community college tuition as 
a percentage of the state average for four-year 
institutions (Kentucky Council on Postsecondary 
Education 2005; Maryland Higher Education 
Commission 2004). However, the issue of 
affordability always includes two aspects — cost 
and access to financial aid — and Kentucky and 
Virginia are the only states that track investment 
in need-based financial aid and average student 
loan debt (Kentucky Council on Postsecondary 
Education 2005; State Council of Higher Education 
for Virginia 2005). 

Academic preparation, moreover, is another key 
factor in providing access to higher education 
for underrepresented groups. The open-door 
admissions policies used by many community 
colleges allow enrollment of students from many 
different academic backgrounds. These students 
may not hold a high school diploma, and even if 
they do, may not have been adequately prepared 
for college-level coursework. Cniy Kentucky, 
Maryland, and Washington include in their 
accountability measures for community colleges 
indicators of the percentage of students needing 
remedial coursework (Kentucky Council on 
Postsecondary Education 2005; Maryland Higher 
Education Commission 2004; Washington Higher 
Education Coordinating Board 2005). California 
and Washington, on the other hand, are the only 
states that are trying to measure the success of 
their English as a Second Language programs, 
a crucial offering for immigrant students at 
community colleges (California Community Colleges 
Board of Governors 2005; Washington Higher 
Education Coordinating Board 2005). 

Measures of Student Success 

A key question for accountability systems is how 
to measure student success. As researchers have 
noted, accountability systems for community 
colleges must define success for a student body 
with a wide range of educational aspirations 
(Boswell 2005; Dougherty & Hong 2005). 

In most states reviewed in this paper, the 
accountability system looks at a combination of 
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persistence, transfer to a four-year institution, 
and degree completion as the principal measure 
of student success. For example, seven out 
of eight states measure student persistence 
from year to year. Six states have indicators 
measuring the number of students transferring 
to four-year institutions and/or the number of 
students achieving transfer requirements. The 
accountability systems in six states measure the 
number of degrees or certificates conferred while, 
in five states, passing rates on state licensure 
exams are measured. 

Since many community colleges attract a large 
number of part-time, non-degree-seeking adult 
learners, analyzing only those indicators relevant 
to traditional degree-seeking students cannot 
truly measure success for the whole student 
population. Measuring success for students who 
may only intend to take a few classes to hone 
their work skills is difficult. No state in this study, 
in fact, had any accountability measures designed 
to assess academic success for such students 
although both Maryland and North Carolina use 
a measure of student satisfaction which includes 
non-completers and might therefore answer 
the question of whether or not these students 
achieved their goals (Maryland Higher Education 
Commission 2004; North Carolina Community 
College System 2005). 

On the other hand, including in the degree 
completion and transfer rates data on students 
who do not intend to earn a degree or to transfer 
will artificially lower those rates. As a result, most 
of the states in this study have developed fairly 
complex ways to determine which students should 
be included in any measure of student success and 
how that success should be defined. In California, 
for example, the “Student Achievement and 
Progress Rate” is defined as: 

Percentage of cohort of first-time students 
with minimum of 12 units earned who 
attempt degree/certificate/transfer threshold 
course within 6 years of entry ... who are 


shown to have achieved ANY of the following 
outcomes or value-added measures of 
progress within 6 years of entry: 

> Earned any A/VAS or Certificate 

> Actual transfer to a four-year institution... 

> Achieved “Transfer Directed” (student 
successfully completed both transfer- 1 eve I 
math AND English courses) 

> Achieved “Transfer Prepared” (student 
successfully completed 60 UC/CSU 
transferable units with a GPA >=2.0 in 
those transferable courses)... 

[> Earned at least 30 units while in the CCC 
system (California Community Colleges 
Board of Governors 2005). 

This sort of measure of student success, while 
attempting to take into consideration the wide 
range of goals embraced by community college 
students, ends up both including many students 
who may not have succeeded in meeting their 
goals and obscuring the fact that students who do 
not meet the cohort definition may actually have 
planned to earn a degree or transfer to a four-year 
institution (McClenney 2005). As such, measures 
like this one may not be effective measures of 
community college performance. Worse still, the 
cohort definition is so complex that it will not be 
clear to many observers, especially those not expert 
in the field of higher education, exactly what is 
being measured by this indicator. 

Most of the states reviewed are also lacking in 
measures of student learning. Kentucky does 
try to measure student engagement in the 
learning experience in order to develop a more 
comprehensive definition of student success and 
intends, in the near future, to include a direct 
measure of student learning in its accountability 
system, perhaps using the Work Keys standardized 
test (Applegate & Noxel 2005). At the moment, 
only Florida uses student performance on a 
statewide learning assessment, the College-Level 


12 


Making Accountability Work 


Academic Skills test, as an accountability measure 
(Florida Community College System 2004). 

A related issue concerns how to measure students’ 
success after leaving a community college. Half 
of the states analyzed for this paper look at the 
success rates of transfers to four-year institutions. 
Washington, for example, considers the three-year 
graduation rate for students who transfer from a 
community college to a public university in the state 
(Washington Higher Education Coordinating Board 
2005). Four states also look at employment rates 
for community college graduates, and some states 
include high-wage employment or employment in a 
field related to the degree earned. Again, however, 
there is less effort to examine the benefit offered 
by the community college to students who do not 
either transfer or complete a degree. 

Mission-Specific Measures 

Community colleges, far more than four-year 
institutions, are called upon to serve a variety of 
educational functions and very diverse student 
populations. The accountability systems examined 
in this paper make an effort to take this challenge 
into account. Nonetheless, the emphasis in most 
of the accountability systems studied remains on 
traditional definitions of success for community 
college students — degree and certificate completion 
or transfer to a four-year institution. Most states 
lack measures that go beyond those indicators to 
examine the many services community colleges 
provide — a problem commonly noted by experts 
on community colleges and accountability (Zumeta 
2000; Dougherty & Hong 2005; Ewell 2005). This 
lack of attention to the range of missions served 
by community colleges offers no way for state 
policymakers to assess the sector’s contributions to 
statewide goals such as improving job training or 
ensuring that students are adequately prepared for 
college-level classes. 

Vocational-technical training is, for example, a 
very important function of community colleges, 
both through degree and certificate programs and 
through short-term workforce training courses. 


Many community colleges were established with 
the intention of providing programs to prepare 
students for specific careers, such as accounting 
and nursing. However, only three states of the 
eight in this study have measures for participation 
in non-credit workforce development programs. 

In terms of for-credit vocational programs, Texas 
and Virginia are the only states that measure the 
number of degrees and certificates awarded in 
priority areas, although California does disaggregate 
community college degrees awarded by program 
(Texas Higher Education Coordinating Board 2005; 
State Council of Higher Education for Virginia 
2005; California Community Colleges Board of 
Governors 2005). 

Community colleges can also play an important 
role in delivering higher education to students who 
may not be able to easily get to a college campus. 
Distance education is an important vehicle to 
accommodate adult learners who have to balance 
work and family responsibilities with school as 
well as to carry college courses to students who, 
for reasons of geography or physical disability, 
may not be able to come to campus. In fact, a 
large percentage of online courses are provided 
by community colleges, and many community 
colleges operate satellite campuses or off-campus 
learning centers to meet the needs of a widely 
dispersed student body. However, no states include 
a distance education measure among those used 
for community colleges, although North Carolina 
does discuss distance education efforts in the more 
contextual part of its accountability report (North 
Carolina Community College System 2005). 

Finally, community colleges are a major resource 
in educating adults whose academic skills 
are not yet at the college level. Community 
colleges routinely offer Adult Basic Education, 

GED preparation, and remedial coursework, 
particularly in math and English. In some states, 
in fact, four-year institutions no longer offer 
remediation at all, requiring instead that their 
students who need such assistance seek it at a 
community college. However, these functions are 
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not well assessed In the statewide accountability 
systems reviewed for this project. Only three 
states used measures to examine community 
college performance in assisting remedial 
students, while only two states measured 
performance in the area of basic skills instruction. 

Attention to Resources 

At one time, performance funding systems, in 
which funding for institutions is directly tied to 
their performance, were preferred by policymakers, 
who believed that colleges and universities could 
be pressured to perform well by the threat of losing 
funding or by the offer of additional funding as 
a reward for good performance. In recent years, 
however, this model has lost much ground. An 
annual survey of State Higher Education Finance 
Officers determined that, between 2001 and 
2003, the number of states using performance 
funding models declined 21 percent from 19 
states to 15 states. During the same period, 
states using performance reporting increased 18 
percent from 39 states to 46. ^ The authors of that 
survey suggest that this change is largely due to 
tight state budgets and the desire of legislatures 
to eliminate programs that are entitled to annual 
budget increases for successful performance (Burke 
& Minassians 2003). 

Our study supports these findings. Of the eight 
states examined in this paper, only two — Florida 
and North Carolina — have higher education 
accountability systems that provide performance 
funding, that is, monies awarded to community 
colleges on the basis of successful performance on 
a defined set of indicators. Florida’s performance- 
based budgeting program, one of several 
accountability systems in the state, awards about 
2 percent of the state’s allocation for community 
colleges — less than $20 million out of a budget 
totaling more than $1.5 billion. Awards granted 
under this system are based on a set of measures 


such as degree completion and success after 
transfer to a four-year institution both for the 
total student body at a given community college 
and for special populations such as Black males 
and economically disadvantaged students (Cisek 
2005). In North Carolina, on the other hand, 
successful performance on certain accountability 
measures mandated by the state legislature allows 
a community college to carry over a small portion 
of its annual budget from one year to the next (K. 
Brown 2005). 

Virginia’s new statewide accountability system 
includes financial incentives but uses a quite 
different model. This accountability system, which 
was only made law in 2005 and has not yet been 
put into effect, came about as a result of a desire 
on the part of the state’s major universities for 
increased autonomy. In a compromise brokered 
by then Governor Mark Warner, institutions that 
meet performance targets receive both increased 
autonomy, such as the right to initiate capital 
projects and set tuition and fees, and financial 
incentives, including interest earned on tuition and 
fees and the right to carry over unexpended funding 
allocations into future years (Massa 2005). 

The remaining states do not have formal 
performance funding models, although some 
certainly use their accountability reports as part of 
the budgeting process. In Maryland, for example, 
the performance of various colleges is a factor 
considered by the Governor’s Budget Office and 
the General Assembly in determining capital 
allocations (Keller 2005). In most states studied, 
however, there is little direct connection between 
performance reporting and the allocation of funding 
for the state’s community colleges. There are few 
repercussions for community colleges that do 
not meet their performance targets. The primary 
motivation for colleges to meet performance 
targets, then, is simply a desire to maintain a 
reputation for quality. 


^ In a performance reporting system, there is no formal connection between an institution's performance and its ongoing funding allocation 
(Burke & Minassians 2003). 
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Given the ongoing budgetary limitations in many 
states, fueled by the economic downturn and 
increases in state spending on Medicaid and K-12 
education, it seems likely that there will not be 
a quick return to the performance funding model 
of accountability. At the same time, however, 
most of the accountability systems we reviewed 
made little or no effort to link resource allocation 
to institutional performance — a connection that 
could help policymakers to better understand the 
needs of the state’s postsecondary institutions 
and to make informed judgments about trade- 
offs between different investment strategies such 
as increasing student aid or building new college 
campuses (Wellman 2002). As a result, these 
systems are not particularly useful for incremental 
budget allocation systems nor are they helping to 
answer larger questions of resource policy. This 
weakness can have a potentially negative effect on 
community colleges as they are asked to shoulder 
more of the responsibility for educating a state’s 
college students. 

Compared to four-year institutions, community 
colleges usually operate with lower levels of 
staffing, heavier teaching loads, less adequate 
physical facilities, and fewer academic resources. 
Recently, budget constraints have further 
exacerbated the problem of providing quality 
education with limited resources. Nonetheless, 
most states in this study do not have the 
measures to gauge the adequacy of community 
college funding, nor do they have measures to 
assess available resources. Most do not analyze 
process indicators such as average class sizes, 
faculty qualifications and retention, library and 
lab space, etc. The only exception is Texas. In its 
accountability system, Texas includes the annual 
appropriations for each community college divided 
by full-time equivalent students and faculty. Texas 
also assesses student/faculty ratios, the percentage 
of faculty with advanced degrees, and average 
class sizes (Texas Higher Education Coordinating 
Board 2005). One other state, Virginia, does have 
a measure for unmet demand for services — a proxy 
for a lack of resources — but this measure could 


also potentially be used to penalize an institution 
that limits enrollments in certain programs (State 
Council of Higher Education for Virginia 2005). 

Higher education accountability has, in recent 
years, moved to a focus on outputs and outcomes 
(Zumeta 2000; Leveille 2005). Such a move is 
not unreasonable given the widespread interest 
in measuring student achievement, but ignoring 
the relation between outputs and resources is 
as bad as simply focusing on inputs, because it 
totally sidesteps the issue of productivity — or the 
cost to produce different levels of outputs. One 
consequence of an exclusive focus on outputs could 
be to lock institutions into avoiding program change 
or improvement, lest they risk investments that 
take years to produce outputs. If, for example, a 
community college hopes to start a new program 
that will consume resources but not produce 
outputs for several years, an accountability system 
that does not make allowances for such a situation 
might push the college to rethink its plan. Similarly, 
a community college might choose not to start a 
GED preparation program, even if one is needed in 
the local community, because that program would 
not be considered in the accountability system. 
Given the crucial role of community colleges as the 
primary source of higher education for the most 
underrepresented groups in the United States, the 
possibility that accountability might limit access 
and services is reason for concern. 

Stability and Usability 

Without stability in accountability measures, 
it is impossible to make effective comparisons 
over time or to take note of important trends 
in a state’s postsecondary educational system. 
Moreover, constant changes in accountability 
measures and data collection requirements place 
a heavy burden on institutions, especially those, 
like community colleges, with limited budgets for 
institutional research. It sometimes seems that 
new accountability systems are put into place 
with every new legislative session, and in some 
states, most notably Florida and California, these 
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accountability systems have been layered on top 
of one another, adding to the confusion about their 
effectiveness (Ewell 2005). 

In fact, most of the accountability systems reviewed 
in this study are new — five were developed within 
the last year or so and four have not yet been 
fully implemented. In California, the legislature 
is considering adding a statewide accountability 
system to the community college accountability 
system just put into place. Because so many of 
these systems are new, our respondents were 
unable to assess their effectiveness or make 
informed guesses as to their future. 

Stability is an important factor in ensuring that an 
accountability system is functioning as expected. 
Those states that have had accountability systems 
in place for a while tend to have established 
procedures for reviewing and updating goals, 
indicators, and benchmarks. In Kentucky, for 
example, the statewide goals are phrased as five key 
questions about the state’s higher education system. 
These goals, which were put into place when the 


accountability system was adopted in 1997, were 
recently reviewed and updated. Two of the original 
questions were replaced by new ones that were felt 
to better assess the state’s current situation, and 
new indicators were added to measure progress 
towards these new goals (Kentucky Council on 
Postsecondary Education 2005). 

Stable accountability systems also seem to 
encourage use of the information by various 
audiences. Maryland’s accountability reports, 
which have been produced since 1988, are, 
for example, regularly used by state officials in 
budgeting and administrative processes (Keller 
2005). In Florida, on the other hand, multiple 
accountability systems compete for audiences, who 
may not even be aware of all of them. Constructive 
use of information technology may also be a means 
to enhance the use of accountability data. Texas’s 
web-based accountability reporting system allows 
the production of reports as Excel spreadsheets, 
which our respondent reports has been a great help 
in answering questions posed by legislative staff 
members (S. Brown 2005). 
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Conclusions 


O ur examination of the effectiveness of 

selected statewide accountability systems in 
meeting the challenges posed by community 
colleges uncovered some hopeful developments. 
Policymakers are developing statewide systems to 
measure the performance of the higher education 
sector. Community colleges play a unique role 
in higher education and will therefore need 
special consideration within these accountability 
systems. The fact that all of the accountability 
systems reviewed have moved away from ranking 
individual colleges and placed new emphasis on 
contextualizing accountability measures through 
benchmarking and peer group comparisons 
indicates a new awareness that accountability 
should focus on meeting state goals not judging 
which institution is best or worst. 

On the other hand, there continue to be significant 
problems with these accountability systems and, 
particularly, with the situation of community 
colleges within them. The frequent disconnect 
we found between performance indicators and 
statewide goals, together with a lack of clarity 
about the appropriate audiences for accountability 
reports, means that cash-strapped community 
colleges may be asked to collect data that are 
not really being used in an effective way to 
drive state policy. The performance indicators 


used for community colleges also continue to be 
problematic with difficulties both in measuring 
the range of missions undertaken by many 
community colleges and in defining appropriate 
measures for student success. Above all, no 
accountability systems we reviewed recognized 
the limited resources available to the state’s 
community colleges or their effective use. As a 
result, policymakers have no way to know if they 
are making the right choices in allocating funds to 
meet the state’s goals for higher education. 

In light of these findings, we recommend that states 
evaluate their existing or proposed accountability 
systems using the six measures defined in this 
paper: focus; differentiation; contextual ization; 
integrity; attention to resources; and stability and 
usability. Doing so will improve the effectiveness of 
their accountability systems, particularly in terms 
of aligning performance indicators with state goals, 
and will help to ensure that community colleges, 
the fastest-growing sector in higher education, 
are appropriately treated in these systems. 
Accountability and transparency are, without a 
doubt, the wave of the future for higher education, 
and state policymakers have the opportunity now 
to build accountability systems that can help them 
use limited resources in effective ways while still 
maintaining quality and access. 
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